Video pattern thumbnails and method

ABSTRACT

An electronic equipment and method for creating a composite image or video pattern thumbnail from portions of data extracted from given frames in a sequence of frames. The extracted portions of frame data vary according to the location of the given frame within the sequence, and the composite image is representative of the content of the media file.

This application is a continuation-in-part of co-pending U.S. patentapplication Ser. No. 11/532,577, filed on Sep. 18, 2006.

TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to video or image processing andelectronic equipment, such as electronic equipment for engaging in voicecommunications.

BACKGROUND

The use of thumbnail images is common in the computer industry.Thumbnails generally are small graphical representations of the contentsof a media file, such as an image file. By viewing the thumbnail, a usermay be able to determine the content of the media file without openingthe file. Conventional thumbnails, however, are less useful when used todepict the contents of a non-image file, such as a video file. Aconventional thumbnail of a video file may be comprised of a singleimage representative of a frame from the video file. Because the videofile may contain tens of thousands of frames or more, a representationof a single frame from the video sometimes is not useful for identifyingthe content of the media file.

SUMMARY

According to one embodiment, an electronic equipment with a controlcircuit having access to a media file, the media file comprising datafor a sequence of frames wherein the data for each frame in the sequenceof frames represents a corresponding image, each frame having a locationrelative to the other frames in the sequence of frames and a processorfor extracting portions of data for each frame in the sequence toproduce a composite image, the location of the portion extracted fromeach given frame varying among the frames.

According to one aspect, the portion extracted from each given framevaries based upon the location of the given frame within the sequence offrames.

According to another aspect, the composite image is comprised of theextracted portions of data arranged in a chronological order accordingto the location of the given frame within the sequence of frames.

According to another aspect, the extracted portions are columns of framedata.

According to another aspect, the data representing the composite imageis comprised of the columns of frame data arranged in a chronologicalorder according to the location of the given frame within the sequenceof frames.

According to yet another aspect, the extracted portions are rows offrame data.

According to another aspect, the data representing the composite imageis comprised of the rows of frame data arranged according to thelocation of the given frame within the sequence of frames.

According to still another aspect, the portion extracted from the framesvaries based upon the location of the greatest changes in thecorresponding images for sequential frames within the sequence offrames.

According to another aspect, the sequence of frames is comprised of moreframes at locations in the media file where there are large changes inthe corresponding images for sequential frames and fewer frames atlocations in the media file where there are small data changes in thecorresponding images for sequential frames.

According to still another aspect the composite image is comprised ofportions of data extracted from areas of the media file where there arelarger changes and smaller changes, wherein the portions extracted fromthe areas having larger changes are brighter than the portions extractedfrom the areas where there are smaller changes.

According to another aspect, the media file is frame-based media.

According to yet another aspect the media file is time-based media.

According to another embodiment, a method of creating an image havingthe steps of accessing a media file, the media file comprising data fora sequence of frames wherein the data represents a corresponding image,each frame having a location relative to the other frames in thesequence of frames and extracting portions of data from the frames inthe sequence to produce a composite image, the location of the extractedportion from each given frame varying among the frames.

According to one aspect, the method the extracting step includes varyingthe extracted portion from each given frame based upon the location ofthe given frame within the sequence of frames.

According to another aspect, the method further includes the step offorming a composite image by arranging the extracted portions in achronological order according to the location of the given frame withinthe sequence of frames.

According to still another aspect, the extracting step includesextracting columns of frame data.

According to another aspect, the method includes the step of arrangingthe columns of frame data in a chronological order according to thelocation of the given frame within the sequence of frames.

According to yet another aspect, the extracting step includes extractingrows of frame data.

According to another aspect, the method further includes the step ofarranging the varying rows of frame data according to the location ofthe given frame within the sequence of frames.

According to another aspect, the extracting step includes extractingdata from more frames at locations in the media file where there arelarge changes in the corresponding images for sequential frames and fromless frames at locations in the media file where there are small changesin the corresponding image for sequential frames.

According to still another aspect, the method further includes the stepof brightening those portions of the composite image formed fromportions of data extracted from the locations in the media file havinglarge changes.

According to another embodiment, a program stored on a machine readablemedium which, when executed by a machine, provides for the creation ofan image by accessing a media file, the media file comprising frame datafor a sequence of frames and extracting varying portions of the framedata from each of the plurality of frames to produce a composite imagecomprised of the extracted portions of frame data from the sequence offrames.

These and further features of the present invention will be apparentwith reference to the following description and attached drawings. Inthe description and drawings, particular embodiments of the inventionhave been disclosed in detail as being indicative of some of the ways inwhich the principles of the invention may be employed, but it isunderstood that the invention is not limited correspondingly in scope.Rather, the invention includes all changes, modifications andequivalents coming within the spirit and terms of the claims appendedhereto.

Features that are described and/or illustrated with respect to oneembodiment may be used in the same way or in a similar way in one ormore other embodiments and/or in combination with or instead of thefeatures of the other embodiments. It should be emphasized that the term“comprises/comprising” when used in this specification is taken tospecify the presence of stated features, integers, steps or componentsbut does not preclude the presence or addition of one or more otherfeatures, integers, steps, components or groups thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an electronic equipment displaying the contents of a medialibrary on a display.

FIG. 2 is a series of frames representative of the content of a mediafile.

FIG. 3 is the series of frames of FIG. 2 used to create a compositeimage.

FIG. 4 is a series of frames representative of the content of a mediafile.

FIG. 5A is the series of frames of FIG. 4 used to create a compositeimage from varying columns of frame data.

FIG. 5B is the series of frames of FIG. 4 used to create a compositeimage from varying rows of frame data.

FIG. 5C is the series of frames of FIG. 4 used to create a compositeimage from varying portions of frame data.

FIG. 5D is a series of frames used to create a composite image.

FIG. 6A is representative of a single frame from a media file.

FIG. 6B is representative of an extracted portion of pixel data from theframe of FIG. 6A.

FIG. 7 is a flow chart representing the steps for processing the mediafile to create the composite image.

FIG. 8 is a an exemplary embodiment of a mobile telephone.

FIG. 9 is a schematic block diagram of the relevant portions of themobile telephone of FIG. 8.

FIG. 10 is a schematic diagram of a communications system in which themobile telephone of FIG. 8 may operate.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following description, with reference to the drawings, likereference numerals are used to refer to like elements throughout. Itwill be appreciated that the drawings are not necessarily to scale. Inthe present application, the invention is described primarily in thecontext of electronic equipment and specifically, a mobile telephone. Itwill be appreciated, however, that the invention is not intended to belimited to a mobile telephone and can be any type of electronicequipment. The term “electronic equipment” includes portable radiocommunication equipment. The term “portable radio communicationequipment,” which hereinafter is referred to as a “mobile radioterminal,” includes all equipment such as mobile telephones, pagers,communicators, electronic organizers, personal digital assistants(PDAs), smartphones, portable communication apparatuses or the like.

An electronic equipment 10 is shown in FIG. 1. The electronic equipment10 is a mobile radio terminal and, in particular, a mobile telephone.Thus, the electronic equipment will be referred to as the mobiletelephone 10.

The functionality of the mobile telephone 10 will be described in moredetail below. Referring briefly to FIG. 9, the mobile telephone 10comprises a control circuit 12, a processor or video processing circuit13 and access to a media library 14. The control circuit 12 is operableto control the video processing circuit 13 to play media files 16 on thedisplay 18 of the mobile telephone 10.

Each media file 16 accessed in or stored by the media library 14 isprogrammed to include information about the media file, such as, forexample, frame data 20 and information regarding the artist, title,format, etc. The media file 16 is comprised of frame data 20representing a plurality of frames for displaying the media file 16 onthe display 18. The frame data 20 is comprised of an array of pixel data22 for each respective frame. The pixel data 22 is processed by thevideo processing circuit 13 to create a graphical representation of theframe on the display 18.

The video processing circuit 13 is operable to process the frame data 20to create a video pattern thumbnail or composite image 24, which iscomprised of a portion of the pixel data 22 for each of the plurality offrames in the media file 16. The composite image 24 therefore representsa unique representation of the content of the media file 16, such as theoverall feel, colorization, light, tempo, etc., of the media file 16.

Referring again to FIG. 1, the contents of the media library 14 (or thefiles to which the media library 14 might have access) are shown in amenu on the display 18. The menu in FIG. 1 is an expandable/collapsiblemenu. In the collapsed state, the menu displays a thumbnail 23 and thetitle 26 of each media file 16. When the menu is expanded, the compositeimage 24, the title 28 of the media file 16 and a progress bar 30 aredisplayed. The progress bar 30 is a graphical depiction of the portionsof the media file 16 that have been accessed, listened to, or played bythe video processing circuit 13, for example, with a media player. Withregard to the media file entitled “File 4.mpg”, the color of theprogress bar 30 has been changed by the video processing circuit 13 toindicate the portions of the file that have been accessed or played. Theportions of the file that have not been played are colored white, whilethe portions of the file that have been played are colored black. Theuser can then easily determine which portions of the file have not beenaccessed and begin viewing the file at that point. If the media file 16is an audio file, the progress bar 30 indicates which chapters orportions have been listened to and which have not, allowing a user tobegin listening to the file where he or she previously stopped.

Although the display 18 of FIG. 1 depicts the thumbnail 23, the filename 28, composite image 24 and a progress bar 30 for each of the mediafiles 16, in the collapsed and/or expanded state, it will be appreciatedthat the control circuit 12 may be programmed control the menu todisplay more, less, or different information, or to customize the menuas desired.

An example of a media file 16, such as a video file, is shown in FIG. 2.The media file 16 is comprised of a plurality of frames 32 arranged inchronological order. Each frame of the media file 16 is comprised offrame data 20 for displaying the frames 32 on the display 18. The framedata 20 is comprised of an array of pixel data 22, which is processed bythe video processing circuit 13 to create a graphical representation ofthe frame 32 on the display 18.

The media files 16, especially video and audio files, may contain largeamounts of data and may be compressed by various compression techniquesthat are well known in the art, including, for example, intraframecompression, interframe compression, lossless compression, etc. If themedia file 16 has been compressed, the frame data 20 for each individualframe may not include a full set of pixel data 22 for representing animage or frame. In other words, the frame data 20 may contain only aportion of the data necessary for showing the frame on the display 18.When the compressed frame data is processed in conjunction with framedata 20 from the rest of the media file 16, however, a full image can beobtained. Thus, it will be appreciated by one of skill in the art that afull image may be obtained from the frame data 20 from any given frameif the frame data 20 is processed in conjunction with additional framedata 20 in the media file 16. The apparatus and processes describedherein are equally applicable to both compressed and uncompressed mediafiles, including video and image files, etc.

Referring to FIG. 3, the video processing circuit 13 is programmed tocreate the composite image 36 by extracting portions of pixel data 22from the media file 16 of FIG. 2. In the embodiment of FIG. 3, a columnof pixel data from each of the plurality of frames 38, 40, 42, 44, 46 inthe video file 16 is extracted to create the composite image 36. Forexample, a column of pixel data 38 a is extracted from frame 38.Likewise, a column of pixel data 40 a is extracted from frame 40, etc.,until columns of pixel data 38 a, 40 a, 42 a, 44 a, 46 a are extractedfrom each of the plurality of frames 38, 40, 42, 44, 46. For purposes ofillustration, the extracted portions of pixel data 38 a, 40 a, 42 a, 44a, 46 a are depicted as shaded regions in FIG. 3; however, it will beappreciated that the frame data 20 and the image represented by theframe data are not altered when the pixel data is extracted.

The video processing circuit 13 is also programmed to arrange theextracted portions 38 a, 40 a, 42 a, 44 a, 46 a of the plurality offrames 38, 40, 42, 44, 46 side-by-side in chronological order to formthe composite image 36. The composite image 36 may then be verticallycompressed to create a reduced-size composite image 48, which may beused in the expandable/collapsible menu of FIG. 1.

Although only a few frames are shown in FIG. 3, the video processingcircuit 13 is operable to extract pixel data 22 from any number offrames 32. For example, the plurality of frames 32 may be determined orbased upon the total number of frames in the media file 16 or video fileand the size of the display 18. For example, if the video file containsframe data 20 for 2400 frames and the display 18 is 240 pixels wide,then the video processing circuit 13 is programmed to extract pixel datafrom 240 even spaced frames, or in this example, every hundredth frameto create the composite image 36. Because the display is 240 pixels wideand data is extracted from 240 frames, one column of pixel data isextracted per frame and for assembling to create the composite image.Thus, the video processing circuit 13 is programmed to extract a singlecolumn of pixel data 22 from each of the 240 frames and combine thepixel data side-by-side in chronological order, as described above, tocreate the composite image 36.

Alternatively, the video processing circuit 13 may be programmed toextract pixel data from 120 frames rather than 240 frames, in which casetwo columns of pixel data would be extracted from each of the pluralityof frames. Similarly, if the display is 480 pixels wide, the pluralityof frames from which pixel data is extracted could be 480 frames, or thecolumns of pixel data could be 4 pixels wide, if data from 120 frameswere extracted.

It will be appreciated that the values used above for the number offrames in the video file and the display size are only illustrative, andthe video processing circuit 13 and the broad concepts described hereinare applicable to media files having any number of frames and fordisplays of any size or resolution.

Generally, the center portion of each video frame in a video filecontains the most relevant subject matter, i.e., it typically is thefocus of the viewer's attention. The pixel data 22 for the upper, lowerand side regions of the frame therefore is less likely to contain themost relevant subject matter. The video processing circuit 13 isprogrammed to extract the pixel data 22 from the center of the framesrather than the pixel data near the borders of the frames to increasethe likelihood that the extracted portions of pixel data will containrelevant portions of the frames. As shown in FIG. 3, the extractedportions 38 a, 40 a, 42 a, 44 a, 46 a of pixel data represent columns offrame data 20 from the center of each of the plurality of frames 38, 40,42, 44, 46. The composite image 36 is therefore formed from pixel data22 that is representative of columns of pixel data that are the mostlikely to contain relevant portions of the video file 16. The videoprocessing circuit 13 also is operable to crop the upper and lowerportions of each column of pixel data as well. Alternatively, the videoprocessing circuit 13 may be programmed to compress the columns of pixeldata to reduce the height of the composite image 36, thereby ensuringthat the composite image is formed from the pixel data likely to containthe most relevant portions of the video file.

In another embodiment, shown in FIG. 4, the media file 16 contains aplurality of frames 50. The frame data for each of the frames may berepresentative of the same general subject matter. For example, as shownin FIG. 4, the media file 16 is a video file of a person's face, forexample, if the person were giving a speech. Because the person's facedoes not substantially move throughout the video, the frame data foreach of the plurality of frames 50 is generally the same, e.g.,representative of the same general picture. Other examples of the mediafiles could be a video of a generally stationary scene in nature, or avideo having several different scenes where the frames comprising eachof the scenes are generally representative of the same image, or anothertype of media file with generally similar subject matter throughout,etc., for each frame in the sequence of frames.

As shown in FIG. 5A, the video processing circuit 13 is programmed toextract a portion of frame data 52 a, 54 a, 56 a, 58 a, 60 a from eachframe in the sequence of frames 52, 54, 56, 58, 60 as described above.The extracted portions of frame data are columns of frame data 52 a, 54a, 56 a, 58 a, 60 a that are extracted from varying positions in eachframe 52, 54, 56, 58, 60 based upon the location of the frame within thesequence of frames. Thus, the extracted portion 52 a from the firstframe 52 in the sequence is on the left side of the frame, the extractedportion 54 a from the second frame 54 in the sequence is located rightof the first extracted portion 52 a. The extracted portion 56 a from thenext frame 56 in the sequence is located right of the location of thesecond extracted portion 54 a, etc.

The sequence of frames 52, 54, 56, 58, 60 are arranged in achronological order. Thus, each frame 52, 54, 56, 58, 60 isrepresentative of a different time in the chronology of the video file.As a result, the location of the extracted portion from each frame inthe sequence of frames progressively shifts from left to right acrossthe frame based upon the location of the frame within the sequence offrames, e.g., the later the frame is in the sequence of frames, thefurther to the right the extracted column is located. The videoprocessing circuit 13 also may be programmed to extract columns of framedata that vary from right to left across the frames, or columns thatvary both left to right and/or right to left, etc.

Regardless of how the columns vary, the video processing circuit 13 isprogrammed to combine the extracted columns 52 a, 54 a, 56 a, 58 a, 60 afrom each of the frames 52, 54, 56, 58, 60 to create a composite image62. If the media file is a video file with substantially similar frames,the composite image 62 would be a rough recreation of the image from oneof the frames. As shown in FIG. 5A, the video pattern thumbnail orcomposite image 62 is pieced together from frame data extracted fromeach frame in the sequence. The resulting image generally isrepresentative of the content of the media file and may be arecognizable image, as shown by the composite image 62 in FIG. 5A. For alarger sequence of frames, this process can be repeated to create aseries of thumbnails arranged side-by-side, analogous to the compositeimage 24 in FIG. 1.

In another embodiment, shown in FIG. 5B, the portions of frame data thatare extracted by the video processing circuit 13 are rows of frame data64 a, 66 a, 68 a, 70 a, 72 a from the sequence of frames 64, 66, 68, 70,72. The rows of frame data are extracted from varying locations withinthe frames. As illustrated in FIG. 5B, the rows vary from the top of theframe to the bottom of the frame based upon the location of the framewithin the sequence of frames. The extracted row 64 a from the firstframe 64 in the sequence of frames is near the top of the frame. Theextracted row 66 a from the next frame 66 in the sequence of frames isbelow the first extracted row 64 a, etc., for each frame in the sequenceof frames. As the frames progress further through the media file, theextracted portion is progressively nearer to the bottom of the frame,until, for example, the bottom row is extracted, or the video processingcircuit 13 cycles to extract rows from the top of the frames, etc.

The video processing circuit 13 is programmed to arrange the extractedrows 64 a, 66 a, 68 a, 70 a, 72 a to form the video pattern thumbnail orcomposite image 74. The first extracted row 64 a is arranged at the topof the composite image 74, the second extracted row 66 a below the firstextracted row 64 a and so on for each row extracted from the sequence offrames. The number of rows of extracted frame data may be based upon thenumber of rows available for displaying the composite image. Forexample, if 30 rows of pixels are available for display of the compositeimage, then 30 rows of frame data may be extracted. For a largersequence of frames, this process can be repeated to create a series ofthumbnails arranged side-by-side, analogous to the composite image 24 inFIG. 1.

In the embodiment of FIG. 5C, the video processing circuit is programmedto extract portions of frame data 76 a, 78 a, 80 a, 82 a, 84 a, 86 afrom each of the frames 76, 78, 80, 82, 84, 86 in the sequence offrames. The blocks of frame data are extracted according to the locationof the frame relative to the sequence. As illustrated, the extractedportion 76 a from the first frame 76 of the sequence is a block from thetop left corner of the frame. The extracted portion 78 a in the nextframe 78 of the sequence is a block from the top right corner of theframe. The extracted portion 80 a from the third frame 80 in thesequence is a block located at the middle left side of the frame, etc.,for each frame in the sequence, as shown in FIG. 5C.

The video processing circuit 13 is programmed to arrange the extractedportions 76 a, 78 a, 80 a, 82 a, 84 a, 86 a to create a composite image88. As shown, the extracted portions are arranged to form the compositeimage based upon the location of the extracted data in the sequence offrames. For example, the first extracted portion 76 a is located at thetop left of the composite image, the second extracted portion 78 a is atthe top right of the composite image, etc., such that the extractedblocks 76 a, 78 a, 80 a, 82 a, 84 a, 86 a create a rough representationof the images from the sequence of frames 76, 78, 80, 82, 84, 86. For alarger sequence of frames, this process can be repeated to create aseries of thumbnails arranged side-by-side, analogous to the compositeimage 24 in FIG. 1.

It will be appreciated that any size portion or block of frame data maybe extracted from each frame. For example, the blocks may be one pixelby one pixel squares or may be half of the frame or larger. It will alsobe appreciated that the embodiment depicted in FIG. 5C is onlyrepresentative of the broad concept of extracting portions or blocks offrame data from the sequence of frames, and the extracted frame data maybe in the form of blocks of frame data or any other shape, such as, forexample, circular shape, rectangular shape, etc.

FIG. 5D depicts another embodiment. As shown in FIG. 5D, the sequence offrames 90, 92, 94, 96, 98 is a series of images having a generally fixedbackground and a focal point of action. For example, as shown in FIG.5D, the media file may be a video of a car traveling down a road suchthat most of the movement in the video is the movement of the car. Thevideo processing circuit 13 is programmed to extract the datarepresentative of the movement in the video file. In order to determinethe location of the most movement in each frame, the video processingcircuit 13 is programmed to compare the data corresponding to sequentialframes in the sequence of frames. For example, when the image of frame90 is compared to the image of subsequent frame 92, the location of thegreatest change in the image of frame 90 is the portion 90 a. Thus,portion 90 a is extracted from frame 90. The processor 13 then comparesthe corresponding images for frame 92 and frame 94. The location of thegreatest change in frame 92 as compared to frame 94 is portion 92 a.This process is repeated for each frame in the sequence of frames untilall of the frames are processed. The extracted portions of data 90 a, 92a, 94 a, 96 a, 98 a are then arranged side-by-side to create thecomposite image 99. Further, as described above, this process may berepeated for any number of frames in the sequence to create thecomposite image 99. It will be appreciated that the processor 13 may beprogrammed to compare the data representing each corresponding images inthe sequence of frames to determine the location of the greatest changesin the media file in the memory 112 (FIG. 9) without displaying eachimage on the display 18.

In another embodiment, the frames that comprise the sequence may bedetermined by the content of the media file. For example, the sequenceof frames may be comprised of more frames at locations in the media filehaving more action or having larger changes in the corresponding imagesfor each frame as compared to the subsequent frame. The composite imagemay then be formed as described in any of the above embodiments. Theresulting composite image would therefore be comprised of data extractedfrom more frames at locations in the media file having more action orgreater changes in the corresponding images and less data extracted fromframes having less action or fewer changes in the corresponding images.

The composite image also may reflect the locations of greater action ormore changes in the corresponding images for the frames in the mediafile. For example, the portions of data in the composite image that areextracted from the locations in the media file having large changes inthe corresponding images or greater action may be brighter than theportions extracted from locations having less action. In addition or inthe alternative, the relative height of the extracted portions may varyaccording to the relative amount of action in media file. For example,the relative height may be increased at those locations in the mediafile where there is more action and/or decreased at those locationswhere there is less action.

It also will be appreciated the in any of the embodiments describedherein, the extracted frame data may vary based upon the location of theframe within the sequence of frames or may vary randomly, or may varyaccording to some other pattern or other arrangement. The compositeimage also may be formed by arranging the extracted portions of framedata randomly.

The varying extracted portions in each of the embodiments of FIGS. 5A-5Dare sequential to one another, e.g., each extracted portion is alignedwith the next extracted portion so that the extracted portions cover theentire frame. It will be appreciated that the extracted portions do notneed to be sequential to one another, e.g., some portions of the framemay not be extracted by the video processing circuit or the extractedportions may be smaller than illustrated in the Figures, or theextracted portions may be spaced apart, etc. The extracted portions,however, may still be arranged in the same general manner to create avideo pattern thumbnail, as described above. It also will be appreciatedthat although illustrated with a limited number of frames, the conceptsdescribed herein are equally applicable to a sequence of any number offrames. Further, even though the composite image has been described asbeing formed from a sequence of similar frames, it will be appreciatedthat the composite image also may be formed from a sequence of framesthat are not similar or in which each frame is different, such as, forexample, the media file 16 depicted in FIG. 2 and described above, or anaudio file, etc.

Although the examples depicted in FIGS. 2-5D are black in white, it willbe appreciated that the description herein is equally applicable tocolor media files. For example, a video file that begins with scenes ina jungle, followed by scenes on a beach or in an ocean and concludeswith scenes in a jungle could have a video pattern thumbnail with shadesof greens and browns, then blues and whites, and then greens and brownsagain. A movie that is generally dark and gory may have a video patternthumbnail that is comprised of pixel data representative of dark colorsand shades of red. In another example, the media file may be an audiofile with the video processing circuit 13 operable to create a graphicalrepresentation of the content of the audio file, such as, for example,frame data representative of a waveform. The video processing circuit 13would then extract the frame data from the plurality of framesrepresenting the waveform to create the composite image, as describedabove.

Referring to FIG. 6A, in another embodiment, the frame data 20 comprisespixel data 22 for multiple colors displayed at the same time, as istypical of a video file or movie. The video processing circuit 13 isoperable to process the frame data 20 from the plurality of frames todetermine the most dominant color for each row of pixel data in thepixel array. The video processing circuit 13 is programmed to extractthe pixel data 22 corresponding to the most dominant color in each rowof frame data 20 to create a vertical column of pixel data. For purposesof simplification in FIG. 6A, several different colors are shown asblocks of color on the display 18. The dominant color of the top portion100 of the display 18 is blue while the dominant color of the bottomportion 101 of the display 18 is green.

FIG. 6B is representative of the column of pixel data created by thevideo processing circuit 13 after determining the most dominant colorsof pixel data in each row of the pixel array. The upper portion 102 ofthe column of pixel data is blue, while the lower portion 103 of thecolumn of pixel data is green. The video processing circuit 13 isprogrammed to process each of the plurality of frames of the video filein same manner. The extracted pixel data is then arranged side-by-sidein chronological order, as described above, to create a composite imagerepresentative of the content of the video or other media file.

It will be appreciated that instead of columns of pixel data, theextracted portions of pixel data may be comprised of rows of pixel data,or squares of pixel data, or random pixel data, etc. Also, instead ofside-by-side chronological order, the extracted portions of pixel datamay be arranged in a checkerboard pattern, randomly, or in anotherpattern or orientation, etc.

Referring now to FIG. 7, a flowchart illustrates the operation of thevideo processing circuit 13 as it is programmed to create the compositeimage. In Step 104, the video processing circuit 13 is programmed toaccess a media file 16 from the media library 14. The media files 16 maybe stored within the memory of the mobile telephone or located remotefrom the mobile telephone 10 and accessed via a radio circuit 122 (FIG.9). Each media file is comprised of a plurality of frames, each havingframe data 20 comprised of an array of pixel data 22 for displaying theframe on the display 18.

In Step 105, the video processing circuit 13 is programmed to extract aportion of the frame data 20 from each of the plurality of frames in themedia file 16. The plurality of frames may be determined by the totalnumber of pixels comprising the width of the display 18. For example, ifthe display is 240 pixels wide, then the video processing circuit 13would be programmed to select 240 frames from the media file 16, or onecolumn of pixel data per pixel column on the display 18. As describedabove, the video processing circuit 13 also may be programmed toaccommodate displays of varying size and resolution and/or video filesof varying lengths. Further, the size of the extracted potions of pixeldata may be dependent on such factors as the resolution of the display,the length of the media file, etc., as described above.

In the extracting step 105, the video processor 13 also may beprogrammed to extract portions of frame data that correspond to the mostmovement in each frame as compared to a subsequent frame in the sequenceof frames. The video processor 13 may be programmed to determine thelocation of the most movement by comparing two adjacent frames anddetermining the location of the most movement, then extract only thatportion of the frame for creating the composite image. The videoprocessor 13 also may be programmed to extract data from a larger numberof frames at locations in the media file having more action and fewerportions of data at those locations in the media file having lessaction.

In Step 105, the video processing circuit 13 is programmed to extract aportion of the pixel data 22 from each of the plurality of frames inStep 104. In one embodiment, the extracted pixel data is comprised ofcolumns of pixel data as described above. In other embodiments, theextracted pixel data may be comprised of rows of pixel data, squares ofpixel data, etc., or any other variation that may be likely to berepresentative the relevant content of the frame data 20 for theselected frame. For example, the extracted portions of data may bevarying columns, rows, blocks or other portions of frame data, asdescribed above.

Referring to Step 106, the video processing circuit 13 is programmed toarrange the extracted pixel data to create the composite image. Thecolumns of extracted pixel data are arranged side-by-side inchronological order to create a composite image that is representativeof the content of the media file 16. The extracted pixel data also maybe arranged in different orientations, such as a horizontal line, or acheckerboard pattern, or randomly, etc. The video processor 13 also maybe programmed to reduce the size of the composite image, or tovertically compress the composite image to create a smaller orreduced-size composite image.

In other embodiments, the video processing circuit 13 is programmed toarrange the varying extracted portions of frame data based upon thelocation of the frames within the sequence of frames. For example, ifthe varying columns of frame data are extracted, the video processingcircuit 13 may be programmed to arrange the columns based upon thelocation of the frame from which the data was extracted within thesequence of frames. Similarly, if rows of frame data are extracted, theyare arranged from top to bottom and/or may be arranged in sets ofextracted data from left to right, such that every time the videoprocessing circuit 13 extracts a row from the top of the frame, a newset is created. Likewise, if blocks or other portions of frame data areextracted, they may be arranged in the same manner, e.g., according tothe location within the sequence of frames.

The video processor also may brighten those portions of extracted datathat correspond to locations in the media file that have more action ordecrease the brightness of those extracted portions that correspond tolocations in the media file having less action. Additionally oralternatively, the video processor 13 may increase or decrease therelative height of the extracted portions in the composite image basedupon the relative level of action in the video file at the locationsfrom which the data is extracted. Thus, at locations corresponding toportions of the media file having more action, the relative height ofthe composite image may be higher than the portions corresponding tothose locations in the video file having less action.

In Step 107, the video processing circuit 13 is programmed to displaythe composite image on the electronic equipment as part of a menu fordisplaying the contents of the media library 14. The display 18 mayinclude the composite image, the name of the media file, the progressbar, etc., such that a user can easily discern the content of the mediafile by looking at the composite image and/or other informationcontained in the menu and shown on the display 18.

Referring to FIG. 8, the mobile telephone 10 also includes a display 18and keypad 110. As is conventional, the display 18 displays informationto a user such as operating state, time, telephone numbers, contactinformation, various navigational menus, etc., which enable the user toutilize the various features of the mobile telephone 10. The display 18is also used to visually display content received by the mobiletelephone 10 and/or retrieved from a memory 112 of the mobile telephone10. In addition, a phone number or a text message entered by the usermay be displayed on the display 18 as part of a graphical user interface(GUI).

The keypad 110 may be conventional in that it provides for a variety ofuser input operations. For example, the keypad 110 typically includesalphanumeric keys 114 for allowing entry of alphanumeric informationsuch as telephone numbers, phone lists, contact information, notes, etc.In addition, the keypad 110 may include special function keys such as a“call send” key for initiating or answering a call, and a “call end” keyfor ending, hanging up, or disconnecting a phone call.

The keypad 110 also may contain special function keys, which may includemenu navigation keys 116, for example, for navigating through a menudisplayed on the display 18, such as the media library 14, contact list,etc., or to select different telephone functions, profiles, settings,etc., as is conventional. Other keys associated with the mobiletelephone 10 may include a volume key, an audio mute key, an on/offpower key, a web browser launch key, a camera key, etc. Keys or key-likefunctionality also may be embodied as a touch screen associated with thedisplay 18. The keys 114 may be used to enter written text (e.g., astring of alphanumeric characters) and to operate GUI menus, or toadjust or modify the mobile telephone settings, such as, for example,the brightness of the display, audio volume, etc.

The mobile telephone 10 includes conventional call circuitry thatenables the mobile telephone 10 to establish a call and/or exchangesignals with a called/calling device, typically another mobile telephoneor landline telephone. The called/calling device need not be anothertelephone, however, it may be some other device such as an Internet webserver, a content providing server, etc.

FIG. 9 represents a functional block diagram of the mobile telephone 10.For the sake of brevity, generally conventional features of the mobiletelephone 10 will not be described in great detail. The mobile telephone10 includes a primary control circuit 12 that is configured to carry outoverall control of the functions and operations of the mobile telephone10. The control circuit 12 may include a processing device 118, such asa CPU, microcontroller or microprocessor. The processing device 118executes code stored in a memory (not shown) within the control circuit12 and/or in a separate memory, such as memory 112, in order to carryout operation of the mobile telephone 10. The memory 112 may be, forexample, one or more of a buffer, a flash memory, a hard drive, aremovable media, a volatile memory, a non-volatile memory or othersuitable device.

It will be apparent to a person having ordinary skill in the art ofcomputer programming, and specifically in applications programming formobile telephones or other electronic devices, how to program a videoprocessing circuit 13 and the mobile telephone 10 to operate and carryout the functions described herein.

Continuing to refer to FIGS. 8 and 9, the mobile telephone 10 includesan antenna 120 coupled to a radio circuit 122. The radio circuit 122includes a radio frequency transmitter and receiver for transmitting andreceiving signals via the antenna 120 as is conventional. The radiocircuit 122 may be configured to operate in a mobile communicationssystem, as well as to receive data and/or audiovisual content or radiotransmissions. For example, the receiver may be an IP datacastcompatible receiver compatible with a hybrid network structure providingmobile communications and digital broadcast services, such as DVB-Hmobile television and/or mobile radio. Other receivers for interactionwith a mobile radio network or broadcasting network are possible andinclude, for example, GSM, CDMA, WCDMA, MBMS, WiFi, WiMax, DVB-H,ISDB-T, etc., as well as advanced versions of these protocols.

The mobile telephone 10 further includes a sound processing circuit 124for processing audio signals transmitted by/received from the radiocircuit 122. Coupled to the sound processing circuit 124 is a speaker(s)126. Also coupled to the sound processing circuit 124 is a microphone128 that enables a user to speak via the mobile telephone 10 as isconventional. The radio circuit 122 and sound processing circuit 124 areeach coupled to the control circuit 12 so as to carry out overalloperation. Audio data may be passed from the control circuit 12 to thesound processing circuit 124 for playback to the user via the speaker126. The audio data may include, for example, audio data from an audiofile stored by the memory 112 and retrieved by the control circuit 12,or an audio file or other media file remotely stored from the mobiletelephone 10 and accessed or received by the radio circuit 122 oranother conventional means. The sound processing circuit 124 may includeany appropriate buffers, decoders, amplifiers, and so forth.

The mobile telephone 10 also includes the aforementioned display 18 andkeypad 110 coupled to the control circuit 12. The display 18 may becoupled to the control circuit 12 by a video processing circuit 13 thatconverts video data to a video signal used to drive the display 18. Thevideo data may be generated by the control circuit 12, retrieved from avideo file that is stored in the memory 112, derived from an incomingvideo data stream received by the radio circuit 122 or obtained by anyother suitable method.

The mobile telephone 10 further includes one or more I/O interface(s)130. The I/O interface(s) 130 may be in the form of typical mobiletelephone I/O interfaces and may include one or more electricalconnectors. As is typical, the I/O interface(s) 130 maybe used to couplethe mobile telephone 10 to a battery charger to charge a battery of apower supply unit (PSU) 132 within the mobile telephone 10. In addition,or in the alternative, the I/O interface(s) 130 may serve to connect themobile telephone 10 to a wired personal hands-free adaptor (not shown),such as a headset to audibly output sound signals output by the soundprocessing circuit 124 to the user. Further, the I/O interface(s) 130may serve to connect the mobile telephone 10 to a personal computer orother device via a data cable. The mobile telephone 10 may receiveoperating power via the I/O interface(s) 130 when connected to a vehiclepower adapter or an electricity outlet power adapter.

The mobile telephone 10 also may include a timer 134 for carrying outtiming functions. Such functions may include timing the durations ofcalls, generating the content of time and date stamps, etc. The mobiletelephone 10 may include a camera 136 for taking digital pictures and/ormovies. Image and/or video files corresponding to the pictures and/ormovies may be stored in the memory 112. The mobile telephone 10 also mayinclude a position data receiver 138, such as a global positioningsystem (GPS) receiver, Galileo satellite system receiver or the like.The mobile telephone 10 also may include a local wireless interface 140,such as an infrared transceiver and/or an RF adaptor (e.g., a Bluetoothadapter), for establishing communication with an accessory, a hands-freeadaptor (e.g., a headset that may audibly output sounds corresponding toaudio data transferred from the mobile telephone 10 to the adapter),another mobile radio terminal, a computer or another device.

The mobile telephone 10 may be configured to transmit, receive andprocess data, such as text messages (e.g., colloquially referred to bysome as “an SMS”), electronic mail messages, multimedia messages (e.g.,colloquially referred to by some as “an MMS”), image files, video files,audio files, ring tones, streaming audio, streaming video, data feeds(including podcasts) and so forth. Processing such data may includestoring the data in the memory 112, executing applications to allow userinteraction with data, displaying video and/or image content associatedwith the data, outputting audio sounds associated with the data and soforth.

The control circuit 12 may be configured to access a media library 14.The media library 14 may access one or more media files 16 stored withinthe memory 112 or located remote from the mobile telephone 10 andaccessed via the radio circuit 122, or another suitable means, as willbe appreciated by one in the art. The media files 16 may be any type ofaudio/visual file, for example including, but not limited to, .avi,.mpeg, .mpeg-2 .mp3, .mp4, .midi, .jpg, .jpeg, .ppt, streaming video,etc. The sound processing circuit 124 may operate to process the contentof the audio files into an audible signal that can be played through thespeaker 126. Likewise, the video processing circuit 13 may operate toprocess the image or video files to produce images or video on thedisplay 18. Both the sound processing circuit 124 and the videoprocessing circuit 13 may be used at the same time for reproducing mediafiles that contain both audio and video coding. The sound and videoprocessing may be performed by any conventional processing circuit ordevice or may be combined into a single processor, as will beappreciated by one of skill in the art.

With additional reference to FIG. 10, the mobile telephone 10 may beconfigured to operate as part of a communications system 142. The system142 may include a communications network 144 having a server 146 (orservers) for managing calls placed by and destined to the mobiletelephone 10, transmitting data to the mobile telephone 10 and carryingout any other support functions. The server 146 communicates with themobile telephone 10 via a transmission medium. The transmission mediummay be any appropriate device or assembly, including, for example, acommunications tower, another mobile telephone, a wireless access point,a wired access point, a satellite, etc. Portions of the network mayinclude wireless transmission pathways. The network 144 may support thecommunications activity of multiple mobile telephones 10, although onlyone mobile telephone 10 is shown in the illustration of FIG. 10.

In one embodiment, the server 146 may operate in stand aloneconfiguration relative to other servers of the communications network144 or may be configured to carry out multiple communications networkfunctions. As will be appreciated, the server 146 may be configured as atypical computer system used to carry out server functions and mayinclude a processor configured to execute software containing logicalinstructions that embody the functions of the server 146.

Although described primarily with respect to frame-based media files(e.g., an mpeg file), it will be appreciated that the concepts describedherein are equally applicable to time-based media files, such as, forexample, scalable vector graphics or SVG files, etc. For example, ratherthan extracting portions of specific frames in a sequence of frames, thevideo processor can be programmed to extract data corresponding todifferent timestamps. The data at each timestamp then can be used tocreate a sequence of frames at arbitrary times throughout the file. Thevideo processor 13 then may extract the data at the representativetimestamps to create the composite image, as described above.

Although described with respect to an electronic equipment, it will alsobe appreciated that the composite image and any associated videoprocessing may be accomplished remote from the electronic equipment, forexample, with a stand-alone computer, server, etc., and such compositeimage may be transferred to the electronic equipment in the informationcontained within the media file or via separate communication mechanism.The term “video pattern thumbnail” may also be used interchangeably withthe term “composite image.”

Although the invention has been shown and described with respect tocertain preferred embodiments, it is obvious that equivalents andmodifications will occur to others skilled in the art upon the readingand understanding of the specification. The present invention includesall such equivalents and modifications, and is limited only by the scopeof the following claims.

1. An electronic equipment comprising: a control circuit having accessto a media file, the media file comprising data for a sequence of frameswherein the data for each frame in the sequence of frames represents acorresponding image, each frame having a location relative to the otherframes in the sequence of frames; a processor for extracting portions ofdata for each frame in the sequence to produce a composite image, thelocation of the portion extracted from each given frame varying amongthe frames.
 2. The electronic equipment of claim 1 wherein the portionextracted from each given frame varies based upon the location of thegiven frame within the sequence of frames.
 3. The electronic equipmentof claim 2 wherein the composite image is comprised of the extractedportions of data arranged in a chronological order according to thelocation of the given frame within the sequence of frames.
 4. Theelectronic equipment of claim 1 wherein the extracted portions arecomprised of columns of frame data.
 5. The electronic equipment of claim4 wherein the data representing the composite image is comprised of thecolumns of frame data arranged in a chronological order according to thelocation of the given frame within the sequence of frames.
 6. Theelectronic equipment of claim 1 wherein the extracted portions arecomprised of rows of frame data.
 7. The electronic equipment of claim 6wherein the data representing the composite image is comprised of therows of frame data arranged according to the location of the given framewithin the sequence of frames.
 8. The electronic equipment of claim 1wherein the portion extracted from the frames varies based upon thelocation of the greatest changes in the corresponding images forsequential frames within the sequence of frames.
 9. The electronicequipment of claim 8 wherein the sequence of frames is comprised of moreframes at locations in the media file where there are large changes inthe corresponding images for sequential frames and fewer frames atlocations in the media file where there are small data changes in thecorresponding images for sequential frames.
 10. The electronic equipmentof claim 9 wherein the composite image is comprised of portions of dataextracted from areas of the media file where there are larger changesand smaller changes, wherein the portions extracted from the areashaving larger changes are brighter than the portions extracted from theareas where there are smaller changes.
 11. The electronic equipment ofclaim 1 wherein the media file is frame-based media.
 12. The electronicequipment of claim 1 wherein the media file is time-based media.
 13. Amethod of creating an image comprising the steps of: accessing a mediafile, the media file comprising data for a sequence of frames whereinthe data represents a corresponding image, each frame having a locationrelative to the other frames in the sequence of frames; and extractingportions of data from the frames in the sequence to produce a compositeimage, the location of the extracted portion from each given framevarying among the frames.
 14. The method of claim 13 wherein theextracting step comprises varying the extracted portion from each givenframe based upon the location of the given frame within the sequence offrames.
 15. The method of claim 14 further comprising the step offorming a composite image by arranging the extracted portions in achronological order according to the location of the given frame withinthe sequence of frames.
 16. The method of claim 13 wherein theextracting step comprises extracting columns of frame data.
 17. Themethod of claim 16 further comprising the step of arranging the columnsof frame data in a chronological order according to the location of thegiven frame within the sequence of frames.
 18. The method of claim 13wherein the extracting step comprises extracting rows of frame data. 19.The method of claim 20 further comprising the step of arranging thevarying rows of frame data according to the location of the given framewithin the sequence of frames.
 20. The method of claim 13 wherein theextracting step comprises extracting data from more frames at locationsin the media file where there are large changes in the correspondingimages for sequential frames and from less frames at locations in themedia file where there are small changes in the corresponding image forsequential frames.
 21. The method of claim 20 further comprising thestep of brightening those portions of the composite image formed fromportions of data extracted from the locations in the media file havinglarge changes.
 22. A program stored on a machine readable medium which,when executed by a machine, provides for the creation of an image by:accessing a media file, the media file comprising frame data for asequence of frames; and extracting varying portions of the frame datafrom each of the plurality of frames to produce a composite imagecomprised of the extracted portions of frame data from the sequence offrames.